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AMENDMENT TO THE CLAIMS 
Please amend the presently pending claims as follows: 

1. (Original) A computer-implemented method for populating an 
electronic form from an electronic image, the method comprising: 

(a) identifying a size, orientation and position of a first 

object having any arbitrary orientation within the 
electronic image; 

(b) identifying information elements from pixels within the 

electronic image that correspond to the first object; 

(c) displaying fields of the electronic form and the 
identified information elements to a user through a 
graphical user interface; 

(d) parsing the information elements into tagged groups of 

different information types; and 

(e) populating the fields of the electronic form with the 

tagged groups to produce a populated form and allowing 
the user to edit the populated fields through the 
graphical user interface. 

2. (Original) The method of claim 1 wherein (a) comprises 
identifying a size, orientation and position of the first object 
among a plurality of objects within the electronic image. 

3. (Original) The method of claim 1 wherein (a) comprises: 

classifying each pixel within the image to produce pixel 

classification data; 
defining an image function to process the pixel 

classification data; 
dividing the image into sub-images based on disparities in 

the image function; and 
processing the sub-images to determine a size, an orientation 

and a position for each of the objects, including the 
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first object. 

4. (Original) The method of claim 3 and further comprising 
repeating the classifying pixels, defining an image function, and 
dividing of the image until the image contains a single object or 
the image can no longer be divided. 

5. (Original) The method of claim 3 wherein classifying 
comprises classifying each pixel as one of a data pixel or a 
background pixel. 

6. (Original) The method of claim 5 and further comprising: 

defining the image function as a sum of data pixels in a 

certain direction; 
calculating the image function in a first direction to 

generate a first data set; 
calculating the image function in a second direction to 

generate a second data set; and 
searching for disparities in the image function in the first 

direction and the image function in the second 

direction. 

7. (Original) The method of claim 1 wherein: 

(b) comprises identifying text blocks and two-dimensional 
locations of the text blocks within the first object 
using optical character recognition; and 

(c) comprises displaying the identified text blocks to the 

user through the graphical user interface 
simultaneously with the fields of the electronic form. 

8. (Original) The method of claim 7 wherein: 

(c) comprises displaying the identified information elements 
within an image of the first object to the user through 



-4- 



the graphical user interface; and 
(d) comprises highlighting the tagged groups of different 
information types within the image of the first object 
with visual indicators that indicate the different 
information types. 

9. (Original) The method of claim 1 wherein: 

(d) comprises receiving the information elements as untagged 

media from an untagged media store, and parsing the 

untagged media to identify information types of the 
information elements; 

(e) comprises automatically populating at least one of the 

fields with at least one of the information elements 
based on the information type of that element; and 
(c) comprises displaying the information elements through an 
object data graphical user interface and displaying the 
populated fields and any unpopulated fields through a 
form graphical user interface. 

10 . (Original) The method of claim 9 wherein (c) further 
comprises employing visual indicator in the object data graphical 
user interface to indicate that an information element is 
compatible with a particular field in the form. 

11 . (Original) The method of claim 9 wherein (c) further 
comprises indicating a status level associated with at least one 
information element in at least one populated field in the form. 

12 . (Original) The method of claim 11 wherein (c) further 
comprises prompting the user to verify and/or correct the at 
least one information element in the at least one populated field 
based on the indicated status level. 
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13. (Original) The method of claim 12 wherein (d) comprises 
updating other fields of the populated form based on the user 
verification and/or correction of the at least one information 
element . 

14 . (Original) The method of claim 1 and further comprising: 

(f) writing side information gleaned from edits made to any 

of the populated fields to a side information store; 
and 

(g) re-parsing the information elements into tagged groups 

of different information types in (d) and re-populating 
at least one other field in (e) based on the side 
information. 

15 . (Original) The method of claim 1 wherein the electronic image 
comprises a plurality of objects and the method further 
comprises: 

(f) performing (a) through (e) for each object, such that, 
for each object, fields of a corresponding electronic 
form are at least partially populated with information 
elements obtained from pixels within that object. 

16. (Original) The method of claim 15 wherein the plurality of 
objects comprises objects of different types. 

17 . (Original) A computer-readable medium comprising computer- 
executable instructions that, when executed by a computer, 
performs a method comprising: 

(a) identifying a size, orientation and position of a first 

object having any arbitrary orientation within an 
electronic image; 

(b) identifying information elements from pixels within the 

electronic image that correspond to the first object; 
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(c) displaying fields of the electronic form and the 
identified information elements to a user through a 
graphical user interface; and 

(d) parsing the information elements into tagged groups of 

different information types; 

(e) populating the fields of the electronic form with the 

tagged groups to produce a populated electronic form 
and allowing the user to edit the populated fields 
through the graphical user interface. 

18 . (Original) The computer-readable medium of claim 17 wherein 
the electronic image comprises a plurality of objects, including 
the first object, and the method further comprises: 

(f) performing (a) through (e) for each object, such that, 

for each object, fields of a corresponding electronic 
form are at least partially populated with information 
elements obtained from pixels within that object. 

19. (Original) The computer-readable medium of claim 18 wherein 
the plurality of objects comprises objects of different types. 

20. (Original) The computer-readable medium of claim 17 wherein 
(a) comprises: 

classifying each pixel within the image to produce pixel 

classification data; 
defining an image function to process the pixel 

classification data; 
dividing the image into sub-images based on disparities in 

the image function; and 
processing the sub-images to determine a size, an orientation 

and a position for each of the objects, including the 

first object. 
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21. (Original) The computer-readable medium of claim 20 wherein 
classifying comprises classifying each pixel as one of a data 
pixel or a background pixel. 

22. (Original) The computer-readable medium of claim 21 and 
further comprising: 

defining the image function as a sum of data pixels in a 

certain direction; 
calculating the image function in a first direction to 

generate a first data set; 
calculating the image function in a second direction to 

generate a second data set; and 
searching for disparities in the image function in the first 

direction and the image function in the second 

direction. 

23. (Original) The computer-readable medium of claim 17 wherein: 

(b) comprises identifying text blocks and two-dimensional 
locations of the text blocks within the first object 
using optical character recognition; 

(c) comprises displaying the identified text blocks within 

an image of the first object to the user through the 
graphical user interface; and 

(d) comprises highlighting the tagged groups of different 

information types within the image of the first object 
with visual indicators that indicate the different 
information types . 

24 . (Original) The computer-readable medium of claim 17 wherein: 

(d) comprises receiving the information elements as untagged 
media from an untagged media store, and parsing the 
untagged media to identify information types of the 
information elements; 
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(e) comprises automatically populating at least one of the 

fields with at least one of the information elements 
based on the information type of that element; and 
(c) comprises displaying the information elements through an 
object data graphical user interface and displaying the 
populated fields and any unpopulated fields through a 
form graphical user interface. 

25 . (Original) The computer-readable medium of claim 24 wherein 
(c) further comprises employing visual indicator in the object 
data graphical user interface to indicate that an information 
element is compatible with a particular field in the form. 

26. (Original) The computer-readable medium of claim 24 wherein 
(c) further comprises indicating a status level associated with 
at least one information element in at least one populated field 
in the form. 

27. (Original) The computer-readable medium of claim 26 wherein 

(c) further comprises prompting the user to verify and/or correct 
the at least one information element in the at least one 
populated field based on the indicated status level. 

28. (Original) The computer-readable medium of claim 27 wherein 

(d) comprises updating other fields of the populated form based 
on the user verification and/or correction of the at least one 
information element. 

29. (Original) The computer-readable medium of claim 17 and 
further comprising: 

(f) writing side information gleaned from edits made by the 

user in step (e) to a side information store; and 

(g) re-parsing the parsing the information elements into 
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tagged groups of different information types in (d) and 
re-populating at least one other field in (e) based on 
the side information. 

30. (Original) A system for at least partially populating 
electronic forms, the system comprising: 

an object detection and extraction module, which processes 
pixels in the electronic image to identify a size, 
orientation and position of an object having any 
arbitrary orientation within the electronic image; 

an optical character recognition module, which identifies 
information elements from pixels within the electronic 
image that correspond to the first object; 

a graphical user interface, which simultaneously displays 
fields of the electronic form and the identified 
information elements to a user; and 

a parsing module, which parses the information elements into 
tagged groups of different information types and at 
least partially populates the fields with the tagged 
groups to produce a populated electronic form. 

31. (Original) The system of claim 30 wherein the graphical user 
interface allows the user to edit the populated fields and any 
unpopulated fields. 

32. (Original) The system of claim 30 wherein the object 
detection and extraction module comprises: 

a data pixel detection module that classifies each pixel of 
the image and defines an image function; 

a segmentation module that is capable of dividing the image 
into smaller sub-images based on disparities in the 
image function; and 

a single object extraction module that processes the sub- 
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images to detect a number, a size, an orientation and a 
position of objects within the image. 

33. (Original) The system of claim 32 wherein the data pixel 
detection module classifies each pixel as one of a data pixel or a 
background pixel. 

-33-34 . (Currently Amended) The system of claim 33 wherein the data 
pixel detection module defines the image function as a sum of data 
pixels in a certain direction, calculates the image function in a 
first direction to generate a first data set, calculates the image 
function in a second direction to generate a second data set, and 
searches for disparities in the image function in the first 
direction and the image function in the second direction. 

■3435. (Currently Amended) The system of claim 30 wherein: 

the optical character recognition module comprises means for 
identifying text blocks and two-dimensional locations 
of the text blocks within the first object using 
optical character recognition; and 
the graphical user interface comprises means for displaying 
the identified text blocks within an image of the first 
object to the user through the graphical user interface 
and highlighting the tagged groups of different 
information types within the image of the first object 
with visual indicators that indicate the different 
information types. 

-3536. (Currently Amended) The system of claim 30 wherein: 

the parsing module comprises means for receiving the 
information elements as untagged media from an untagged 
media store, parsing the untagged media to identify 
information types of the information elements, and 
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automatically populating at least one of the fields 
with at least one of the information elements based on 
the information type of that element; and 
the graphical user interface comprises means for displaying 
the information elements through an object data 
graphical user interface and displaying the populated 
fields and any unpopulated fields through a form 
graphical user interface. 

■3437 . (Currently Amended) The system of claim 35 wherein the 
object data graphical user interface comprises a visual indicator 
to indicate that an information element is compatible with a 
particular field in the form. 

37-38 . (Currently Amended) The system of claim 35 wherein the form 
graphical user interface comprises a status level indicator 
associated with at least one information element in at least one 
populated field in the form. 

-3-8-39 . (Currently Amended) The system of claim 37 wherein the 
graphical user interface comprises means for prompting the user 
to verify and/or correct at least one information element in at 
least one populated field based on the status level indicator 
associated with that field. 

•34*4 0 . (Currently Amended) The system of claim 31 wherein the 
parsing module comprises means for updating other fields of the 
populated form according to edits made by the user to the 
populated fields and the unpopulated fields. 

4341. (Currently Amended) A method for populating electronic forms 
from an electronic image having first and second objects of 
different information types, the method comprising: 
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identifying a size, orientation and position of the first 
and second objects within the electronic image and 
dividing the electronic image into sub-images 
corresponding to pixels in the electronic image 
associated with the size, orientation and position of 
each object; 

performing optical character recognition on each sub-image 
to identify untagged information elements within the 
corresponding object; 

for each sub-image, parsing the untagged information 
elements into tagged information elements; 

populating fields in a first electronic form type with the 
tagged information elements identified from the sub- 
image of the first object to produce a first populated 
form; 

populating fields in a second electronic form type with the 
tagged information elements identified from the sub- 
image of the second object to produce a second 
populated form; and 

displaying the first and second populated forms and the 
untagged information elements to a user through a 
graphical user interface and allowing the user to edit 
the first and second populated forms through the 
graphical user interface. 

4-3r42 . (Currently Amended) The method of claim 40 wherein: 

the first object comprises a business card and the second 

object comprises a purchase receipt; and 
the first electronic form comprises a contact record of a 

software address book and the second electronic form 

comprises an electronic financial record of a software 

financial application. 



